<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.9.1"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Eigen-unsupported: Non linear optimization module</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
  $(document).ready(function() { init_search(); });
/* @license-end */
</script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
</script>
<script type="text/javascript" async="async" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
<link href="eigendoxy.css" rel="stylesheet" type="text/css">
<!--  -->
<script type="text/javascript" src="eigen_navtree_hacks.js"></script>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="Eigen_Silly_Professor_64x64.png"/></td>
  <td id="projectalign" style="padding-left: 0.5em;">
   <div id="projectname"><a href="http://eigen.tuxfamily.org">Eigen-unsupported</a>
   &#160;<span id="projectnumber">3.4.90 (git rev 67eeba6e720c5745abc77ae6c92ce0a44aa7b7ae)</span>
   </div>
  </td>
   <td>        <div id="MSearchBox" class="MSearchBoxInactive">
        <span class="left">
          <img id="MSearchSelect" src="search/mag_sel.svg"
               onmouseover="return searchBox.OnSearchSelectShow()"
               onmouseout="return searchBox.OnSearchSelectHide()"
               alt=""/>
          <input type="text" id="MSearchField" value="Search" accesskey="S"
               onfocus="searchBox.OnSearchFieldFocus(true)" 
               onblur="searchBox.OnSearchFieldFocus(false)" 
               onkeyup="searchBox.OnSearchFieldChange(event)"/>
          </span><span class="right">
            <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.svg" alt=""/></a>
          </span>
        </div>
</td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.9.1 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
var searchBox = new SearchBox("searchBox", "search",false,'Search','.html');
/* @license-end */
</script>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
  <div id="nav-tree">
    <div id="nav-tree-contents">
      <div id="nav-sync" class="sync"></div>
    </div>
  </div>
  <div id="splitbar" style="-moz-user-select:none;" 
       class="ui-resizable-handle">
  </div>
</div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
$(document).ready(function(){initNavTree('group__NonLinearOptimization__Module.html',''); initResizable(); });
/* @license-end */
</script>
<div id="doc-content">
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div class="header">
  <div class="summary">
<a href="#nested-classes">Classes</a>  </div>
  <div class="headertitle">
<div class="title">Non linear optimization module</div>  </div>
</div><!--header-->
<div class="contents">
<a name="details" id="details"></a><h2 class="groupheader">Detailed Description</h2>
<div class="fragment"><div class="line"><span class="preprocessor">#include &lt;unsupported/Eigen/NonLinearOptimization&gt;</span></div>
</div><!-- fragment --><p>This module provides implementation of two important algorithms in non linear optimization. In both cases, we consider a system of non linear functions. Of course, this should work, and even work very well if those functions are actually linear. But if this is so, you should probably better use other methods more fitted to this special case.</p>
<p>One algorithm allows to find a least-squares solution of such a system (Levenberg-Marquardt algorithm) and the second one is used to find a zero for the system (Powell hybrid "dogleg" method).</p>
<p>This code is a port of minpack (<a href="http://en.wikipedia.org/wiki/MINPACK">http://en.wikipedia.org/wiki/MINPACK</a>). Minpack is a very famous, old, robust and well renowned package, written in fortran. Those implementations have been carefully tuned, tested, and used for several decades.</p>
<p>The original fortran code was automatically translated using f2c (<a href="http://en.wikipedia.org/wiki/F2c">http://en.wikipedia.org/wiki/F2c</a>) in C, then c++, and then cleaned by several different authors. The last one of those cleanings being our starting point : <a href="http://devernay.free.fr/hacks/cminpack.html">http://devernay.free.fr/hacks/cminpack.html</a></p>
<p>Finally, we ported this code to <a class="el" href="namespaceEigen.html" title="Namespace containing all symbols from the Eigen library.">Eigen</a>, creating classes and API coherent with <a class="el" href="namespaceEigen.html" title="Namespace containing all symbols from the Eigen library.">Eigen</a>. When possible, we switched to <a class="el" href="namespaceEigen.html" title="Namespace containing all symbols from the Eigen library.">Eigen</a> implementation, such as most linear algebra (vectors, matrices, stable norms).</p>
<p>Doing so, we were very careful to check the tests we setup at the very beginning, which ensure that the same results are found.</p>
<h1><a class="anchor" id="Tests"></a>
Tests</h1>
<p>The tests are placed in the file unsupported/test/NonLinear.cpp.</p>
<p>There are two kinds of tests : those that come from examples bundled with cminpack. They guaranty we get the same results as the original algorithms (value for 'x', for the number of evaluations of the function, and for the number of evaluations of the Jacobian if ever).</p>
<p>Other tests were added by myself at the very beginning of the process and check the results for Levenberg-Marquardt using the reference data on <a href="http://www.itl.nist.gov/div898/strd/nls/nls_main.shtml">http://www.itl.nist.gov/div898/strd/nls/nls_main.shtml</a>. Since then i've carefully checked that the same results were obtained when modifying the code. Please note that we do not always get the exact same decimals as they do, but this is ok : they use 128bits float, and we do the tests using the C type 'double', which is 64 bits on most platforms (x86 and amd64, at least). I've performed those tests on several other implementations of Levenberg-Marquardt, and (c)minpack performs VERY well compared to those, both in accuracy and speed.</p>
<p>The documentation for running the tests is on the wiki <a href="http://eigen.tuxfamily.org/index.php?title=Tests">http://eigen.tuxfamily.org/index.php?title=Tests</a></p>
<h1><a class="anchor" id="API"></a>
API: overview of methods</h1>
<p>Both algorithms needs a functor computing the Jacobian. It can be computed by hand, using auto-differentiation (see <a class="el" href="group__AutoDiff__Module.html">Auto Diff module</a>), or using numerical differences (see <a class="el" href="group__NumericalDiff__Module.html">Numerical differentiation module</a>). For instance: </p><div class="fragment"><div class="line">MyFunc func;</div>
<div class="line">NumericalDiff&lt;MyFunc&gt; func_with_num_diff(func);</div>
<div class="line">LevenbergMarquardt&lt;NumericalDiff&lt;MyFunc&gt; &gt; lm(func_with_num_diff);</div>
</div><!-- fragment --><p> For HybridNonLinearSolver, the method solveNumericalDiff() does the above wrapping for you.</p>
<p>The methods LevenbergMarquardt.lmder1()/lmdif1()/lmstr1() and HybridNonLinearSolver.hybrj1()/hybrd1() are specific methods from the original minpack package that you probably should NOT use until you are porting a code that was previously using minpack. They just define a 'simple' API with default values for some parameters.</p>
<p>All algorithms are provided using two APIs :</p><ul>
<li>one where the user inits the algorithm, and uses '*OneStep()' as much as he wants : this way the caller have control over the steps</li>
<li>one where the user just calls a method (optimize() or solve()) which will handle the loop: init + loop until a stop condition is met. Those are provided for convenience.</li>
</ul>
<p>As an example, the method LevenbergMarquardt::minimize() is implemented as follow: </p><div class="fragment"><div class="line">Status LevenbergMarquardt&lt;FunctorType,Scalar&gt;::minimize(FVectorType  &amp;x, <span class="keyword">const</span> <span class="keywordtype">int</span> mode)</div>
<div class="line">{</div>
<div class="line">    Status status = minimizeInit(x, mode);</div>
<div class="line">    <span class="keywordflow">do</span> {</div>
<div class="line">        status = minimizeOneStep(x, mode);</div>
<div class="line">    } <span class="keywordflow">while</span> (status==Running);</div>
<div class="line">    <span class="keywordflow">return</span> status;</div>
<div class="line">}</div>
</div><!-- fragment --><h1><a class="anchor" id="examples"></a>
Examples</h1>
<p>The easiest way to understand how to use this module is by looking at the many examples in the file unsupported/test/NonLinearOptimization.cpp. </p>
<table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a name="nested-classes"></a>
Classes</h2></td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classEigen_1_1HybridNonLinearSolver.html">Eigen::HybridNonLinearSolver&lt; FunctorType, Scalar &gt;</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight">Finds a zero of a system of n nonlinear functions in n variables by a modification of the Powell hybrid method ("dogleg").  <a href="classEigen_1_1HybridNonLinearSolver.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classEigen_1_1LevenbergMarquardt.html">Eigen::LevenbergMarquardt&lt; FunctorType_ &gt;</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight">Performs non linear optimization over a non-linear function, using a variant of the Levenberg Marquardt algorithm.  <a href="classEigen_1_1LevenbergMarquardt.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table>
</div><!-- contents -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
  <ul>
    <li class="footer">Generated on Thu Apr 21 2022 13:08:00 for Eigen-unsupported by
    <a href="http://www.doxygen.org/index.html">
    <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.9.1 </li>
  </ul>
</div>
</body>
</html>
